Empirical Comparison of Nonparametric Regression Estimates on Real Data

The performance of nine different nonparametric regression estimates is empirically compared on ten different real data sets. The number of data points in the real data sets varies between 7900 and 18000, where each real data set contains between 5 and 20 variables. The nonparametric regression estimates include kernel, partitioning, nearest neighbor, additive spline, neural network, penalized smoothing splines, local linear kernel, regression trees and random forests estimates. The main result is a table containing the empirical L2 risks of all nine nonparametric regression estimates on the evaluation part of the different data sets. The neural networks and random forests are the two estimates performing best. The data sets are publicly available, so that any new regression estimate can be easily compared with all nine estimates considered in this paper by just applying it to the publicly available data and by computing its empirical L2 risks on the evaluation part of the data sets. AMS classification: Primary 62G08, secondary 62P99.